Search Results: "jeff"

31 August 2021

Russ Allbery: kstart 4.3

kstart provides the programs k5start and krenew, which are similar to the Kerberos kinit program with some extra support for running programs with separate credentials and running as a daemon. This is the first full release in nearly six years. The major change is new support for the Linux kafs module, which is a native Linux implementation of the AFS protocol that David Howells and others have been working on for years. It has an entirely different way of thinking about tokens and credential isolation built on Linux keyrings rather than the AFS token concept (which sometimes uses keyrings, but in a different way, and sometimes uses other hacks). k5start and krenew, when run with the -t option to get AFS tokens, would fail if AFS was not available. That meant -t would fail with kafs even if the AKLOG environment variable were set properly to aklog-kafs. This release fixes that. The programs also optionally link with libkeyutils and use it when used to run a command to isolate the AFS credentials from the calling process. This is done by creating a new session keyring and linking it to the user keyring before running the aklog program. Thanks to Bill MacAllister, David Howells, and Jeffrey Altman for the help with this feature. I'm not sure that I have it right, so please let me know if it doesn't work for you. Also in this release is a fix from Aasif Versi to use a smarter exit status if k5start or krenew is running another program and that program is killed with a signal. Previously, that would cause k5start or krenew to exit with a status of 0, which was not helpful. Now it exits with a status formed by adding 128 to the signal number, which matches the behavior of bash. Since this is the first release in a while, it also contains some other minor fixes and portability updates. You can get the latest release from the kstart distribution page.

24 July 2021

Dirk Eddelbuettel: littler 0.3.13: Moar Goodies

max-heap image The fourteenth release of littler as a CRAN package just landed, following in the now fifteen year history (!!) as a package started by Jeff in 2006, and joined by me a few weeks later. littler is the first command-line interface for R as it predates Rscript. It allows for piping as well for shebang scripting via #!, uses command-line arguments more consistently and still starts faster. It also always loaded the methods package which Rscript only started to do in recent years. littler lives on Linux and Unix, has its difficulties on macOS due to yet-another-braindeadedness there (who ever thought case-insensitive filesystems as a default were a good idea?) and simply does not exist on Windows (yet the build system could be extended see RInside for an existence proof, and volunteers are welcome!). See the FAQ vignette on how to add it to your PATH. A few examples are highlighted at the Github repo, as well as in the examples vignette. This release brings two new example scripts and command wrappers (compiledDeps.r, silenceTwitterAccount.r), along with extensions, corrections, or polish for a number a of other examples as detailed in the NEWS file entry below.

Changes in littler version 0.3.13 (2021-07-24)
  • Changes in examples
    • New script compiledDeps.r to show which dependencies are compiled
    • New script silenceTwitterAccount.r wrapping rtweet
    • The -c or --code option for installRSPM.r was corrected
    • The kitten.r script now passes options bunny and puppy on to the pkgKitten::kitten() call; new options to call the Arma and Eigen variants were added
    • The getRStudioDesktop.r and getRStudioServer.r scripts were updated for a change in rvest
    • Two typos in the tt.r help message were correct (Aaron Wolen in #86)
    • The message in cranIncoming.r was corrected.
  • Changes in package
    • Added Continuous Integration runner via run.sh from r-ci.
    • Two vignettes got two extra vignette attributes.
    • The mkdocs-material documentation input was moved.
    • The basic unit tests were slightly refactored and updated.

My CRANberries provides a comparison to the previous release. Full details for the littler release are provided as usual at the ChangeLog page, and now also on the new package docs website. The code is available via the GitHub repo, from tarballs and now of course also from its CRAN page and via install.packages("littler"). Binary packages are available directly in Debian as well as soon via Ubuntu binaries at CRAN thanks to the tireless Michael Rutter. Comments and suggestions are welcome at the GitHub repo. If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

29 March 2021

Jamie McClelland: The problem with Richard Stallman is not about free speech

Free speech and censorship are critically important issues. And, using them to defend Richard Stallman's return to the Free Software Foundation (FSF) board is just plain wrong. Richard Stallman resigned from the Board in 2019 after he sent an email in defense of Marvin Minsky (Minsky is accused of raping one of Jeffreys Epstein's victims). Stallman's fateful email, however, is just one piece of the reason for why he should not be on the board. The full story is about his history of abuse toward women and is extensive. On March 21st, 2021, Stallman announced he is back on the board. There are profound reasons why any movement interested in equitable and open participation would want to publicly distance themselves from Stallman. However, the long form defenses of Stallman, including a note from Nadine Strossen, the former executive director of the ACLU, quoted in this defense, persist. Many of the arguments defending Richard Stallman (including the one from Strossen) are grounded in a belief that Stallman is being punished for his unpopular political views, which deserve to be defended on the grounds of freedom of expression. That's wrong. Stallman should be kicked off the board because he has a long history of abusing his position to hit on women, which, when combined with his public opinions on under-age sex and his defense of Minsky, send a strong signal that the FSF does not care about the participation of women. Being on a board of directors is a privilege, not a right. Being removed from a board is not a punnishment. And being criticized and removed from a board because your behavior and public statements are an obstacle to building an inclusive and equitable movement is what every board should strive to do. If we are going to make this an issue about free expression, it should be about all the political expression lost to the free software movement because Stallman's unequal behavior toward women excluded an enormous number of talented individuals.

28 March 2021

Norbert Preining: RMS, Debian, and the world

Too much has been written, a war of support letters is going on, Debian is heading head first like Lemmings into the same war. And then, there is this by the first female President of the American Civil Liberties Union (ACLU): Civil Rights Activist Nadine Strossen s Response To #CancelStallman:
I find it so odd that the strong zeal for revenge and punishment if someone says anything that is perceived to be sexist or racist or discriminatory comes from liberals and progressives. There are so many violations [in cases like Stallman s] of such fundamental principles to which progressives and liberals cling in general as to what is justice, what is fairness, what is due process.
One is proportionality: that the punishment should be proportional to the offense. Another one is restorative justice: that rather than retribution and punishment, we should seek to have the person constructively come to understand, repent, and make amends for an infraction. Liberals generally believe society to be too punitive, too harsh, not forgiving enough. They are certainly against the death penalty and other harsh punishments even for the most violent, the mass murderers. Progressives are right now advocating for the release of criminals, even murderers. To then have exactly the opposite attitude towards something that certainly is not committing physical violence against somebody, I don t understand the double standard!
Another cardinal principle is we shouldn t have any guilt by association. [To hold culpable] these board members who were affiliated with him and ostensibly didn t do enough to punish him for things that he said which by the way were completely separate from the Free Software Foundation is multiplying the problems of unwarranted punishment. It extends the punishment where the argument for responsibility and culpability becomes thinner and thinner to the vanishing point. That is also going to have an enormous adverse impact on the freedom of association, which is an important right protected in the U.S. by the First Amendment.
The Supreme Court has upheld freedom of association in cases involving organizations that were at the time highly controversial. It started with NAACP (National Association for the Advancement of Colored People) during the civil rights movement in the 1950s and 60s, but we have a case that s going to the Supreme Court right now regarding Black Lives Matter. The Supreme Court says even if one member of the group does commit a crime in both of those cases physical violence and assault that is not a justification for punishing other members of the group unless they specifically intended to participate in the particular punishable conduct.
Now, let s assume for the sake of argument, Stallman had an attitude that was objectively described as discriminatory on the basis on race and gender (and by the way I have seen nothing to indicate that), that he s an unrepentant misogynist, who really believes women are inferior. We are not going to correct those ideas, to enlighten him towards rejecting them and deciding to treat women as equals through a punitive approach! The only approach that could possibly work is an educational one! Engaging in speech, dialogue, discussion and leading him to re-examine his own ideas.
Even if I strongly disagree with a position or an idea, an expression of an idea, advocacy of an idea, and even if the vast majority of the public disagrees with the idea and finds it offensive, that is not a justification for suppressing the idea. And it s not a justification for taking away the equal rights of the person who espouses that idea including the right to continue holding a tenured position or other prominent position for which that person is qualified.
But a number of the ideas for which Richard Stallman has been attacked and punished are ideas that I as a feminist advocate of human rights find completely correct and positive from the perspective of women s equality and dignity! So for example, when he talks about the misuse and over use and flawed use of the term sexual assault, I completely agree with that critique! People are indiscriminantly using that term or synonyms to describe everything from the most appaulling violent abuse of helpless vulnerable victims (such as a rape of a minor) to any conduct or expression in the realm of gender or sexuality that they find unpleasant or disagreeable.
So we see the term sexual assault and sexual harrassment used for example, when a guy asks a woman out on a date and she doesn t find that an appealing invitation. Maybe he used poor judgement in asking her out, maybe he didn t, but in any case that is NOT sexual assault or harassment. To call it that is to really demean the huge horror and violence and predation that does exist when you are talking about violent sexual assault. People use the term sexual assault/ sexual harassment to refer to any comment about gender or sexuality issues that they disagree with or a joke that might not be in the best taste, again is that to be commended? No! But to condemn it and equate it with a violent sexual assault again is really denying and demeaning the actual suffering that people who are victims of sexual assault endure. It trivializes the serious infractions that are committed by people like Jeffrey Epstein and Harvey Weinstein. So that is one point that he made that I think is very important that I strongly agree with.
Secondly and relatedly, [Richard Stallman] never said that he endorse child pornography, which by definition the United States Supreme Court has defined it multiple times is the sexual exploitation of an actual minor. Coerced, forced, sexual activity by that minor, with that minor that happens to be filmed or photographed. That is the definition of child pornography. He never defends that! What the point he makes, a very important one, which the U.S. Supreme Court has also made, is mainly that we overuse and distort the term child pornography to refer to any depiction of any minor in any context that is even vaguely sexual.
So some people have not only denounced as child pornography but prosecuted and jailed loving devoted parents who committed the crime of taking a nude or semi-nude picture of their own child in a bathtub or their own child in a bathing suit. Again it is the hysteria that has totally refused to draw an absolutely critical distinction between actual violence and abuse, which is criminal and should be criminal, to any potentially sexual depiction of a minor. And I say potentially because I think if you look at a picture a parent has taken of a child in a bathtub and you see that as sexual, then I d say there s something in your perspective that might be questioned or challenged! But don t foist that upon the parent who is lovingly documenting their beloved child s life and activities without seeing anything sexual in that image.
This is a decision that involves line drawing. We tend to have this hysteria where once we hear terms like pedophilia of course you are going to condemn anything that could possibly have that label. Of course you would. But societies around the world throughout history various cultures and various religions and moral positions have disagreed about at what age do you respect the autonomy and individuality and freedom of choice of a young person around sexuality. And the U.S. Supreme Court held that in a case involving minors right to choose to have an abortion.
By the way, [contraception and abortion] is a realm of sexuality where liberals and progressives and feminists have been saying, Yes! If you re old enough to have sex. You should have the right to contraception and access to it. You should have the right to have an abortion. You shouldn t have to consult with your parents and have their permission or a judge s permission because you re sufficiently mature. And the Supreme Court sided in accord of that position. The U.S. Supreme Court said constitutional rights do not magically mature and spring into being only when someone happens to attain the state defined age of majority.
In other words the constitution doesn t prevent anyone from exercising rights, including Rights and sexual freedoms, freedom of choice and autonomy at a certain age! And so you can t have it both ways. You can t say well we re strongly in favor of minors having the right to decide what to do with their own bodies, to have an abortion what is in some people s minds murder but we re not going to trust them to decide to have sex with somewhat older than they are.
And I say somewhat older than they are because that s something where the law has also been subject to change. On all issues of when you obtain the age of majority, states differ on that widely and they choose different ages for different activities. When you re old enough to drive, to have sex with someone around your age, to have sex with someone much older than you. There is no magic objective answer to these questions. I think people need to take seriously the importance of sexual freedom and autonomy and that certainly includes women, feminists. They have to take seriously the question of respecting a young person s autonomy in that area.
There have been famous cases of 18 year olds who have gone to prison because they had consensual sex with their girlfriends who were a couple of years younger. A lot of people would not consider that pedophilia and yet under some strict laws and some absolute definitions it is. Romeo and Juliet laws make an exception to pedophilia laws when there is only a relatively small age difference. But what is relatively small? So to me, especially when he says he is re-examining his position, Stallman is just thinking through the very serious debate of how to be protective and respectful of young people. He is not being disrespectful, much less wishing harm upon young people, which seems to be what his detractors think he s doing.
Unfortunately, I don t think the Anti-Harassment Team of Debian and others of the usual group of warriors will ever read less understand what is written there. So sad.

7 January 2021

Russell Coker: Monopoly the Game

The Smithsonian Mag has an informative article about the history of the game Monopoly [1]. The main point about Monopoly teaching about the problems of inequality is one I was already aware of, but there are some aspects of the history that I learned from the article. Here s an article about using modified version of Monopoly to teach Sociology [2]. Maria Paino and Jeffrey Chin wrote an interesting paper about using Monopoly with revised rules to teach Sociology [3]. They publish the rules which are interesting and seem good for a class. I think it would be good to have some new games which can teach about class differences. Maybe have an Escape From Poverty game where you have choices that include drug dealing to try and improve your situation or a cooperative game where people try to create a small business. While Monopoly can be instructive it s based on the economic circumstances of the past. The vast majority of rich people aren t rich from land ownership.
Related posts:
  1. War On Terror the game TerrorBullGames has just released the board game War On Terror....
  2. Links February 2020 Truthout has an interesting summary of the US Wars Without...
  3. Links December 2013 Andres Lozano gave an interesting TED talk about the use...

8 December 2020

Russell Coker: Links December 2020

Business Insider has an informative article about the way that Google users can get locked out with no apparent reason and no recourse [1]. Something to share with clients when they consider putting everything in the cloud . Vice has an interestoing article about people jailbreaking used Teslas after Tesla has stolen software licenses that were bought with the car [2]. The Atlantic has an interesting article titled This Article Won t Change Your Mind [3]. It s one of many on the topic of echo chambers but has some interesting points that others don t seem to cover, such as regarding the benefits of groups when not everyone agrees. Inequality.org has lots of useful information about global inequality [4]. Jeffrey Goldberg has an insightful interview with Barack Obama for the Atlantic about the future course of American politics and a retrospective on his term in office [5]. A Game Designer s Analysis Of QAnon is an insightful Medium article comparing QAnon to an augmented reality game [6]. This is one of the best analysis of QAnon operations that I ve seen. Decrypting Rita is one of the most interesting web comics I ve read [7]. It makes good use of side scrolling and different layers to tell multiple stories at once. PC Mag has an article about the new features in Chrome 87 to reduce CPU use [8]. On my laptop I have 1/3 of all CPU time being used when it is idle, the majority of which is from Chrome. As the CPU has 2 cores this means the equivalent of 1 core running about 66% of the time just for background tabs. I have over 100 tabs open which I admit is a lot. But it means that the active tabs (as opposed to the plain HTML or PDF ones) are averaging more than 1% CPU time on an i7 which seems obviously unreasonable. So Chrome 87 doesn t seem to live up to Google s claims. The movie Bad President starring Stormy Daniels as herself is out [9]. Poe s Law is passe. Interesting summary of Parler, seems that it was designed by the Russians [10]. Wired has an interesting article about Indistinguishability Obfuscation, how to encrypt the operation of a program [11]. Joerg Jaspert wrote an interesting blog post about the difficulties packagine Rust and Go for Debian [12]. I think that the problem is many modern languages aren t designed well for library updates. This isn t just a problem for Debian, it s a problem for any long term support of software that doesn t involve transferring a complete archive of everything and it s a problem for any disconnected development (remote sites and sites dealing with serious security. Having an automatic system for downloading libraries is fine. But there should be an easy way of getting the same source via an archive format (zip will do as any archive can be converted to any other easily enough) and with version numbers.
Related posts:
  1. Links November 2020 KDE has a long term problem of excessive CPU time...
  2. Links January 2020 C is Not a Low Level Language [1] is an...
  3. Links February 2020 Truthout has an interesting summary of the US Wars Without...

8 October 2020

Dirk Eddelbuettel: littler 0.3.12: Exciting updates

max-heap image The thirteenth release of littler as a CRAN package became available today (after a three day rest at CRAN for no real reason), following in the fourteen-ish year history as a package started by Jeff in 2006, and joined by me a few weeks later. littler is the first command-line interface for R as it predates Rscript. It allows for piping as well for shebang scripting via #!, uses command-line arguments more consistently and still starts faster. It also always loaded the methods package which Rscript only started to do in recent years. littler lives on Linux and Unix, has its difficulties on macOS due to yet-another-braindeadedness there (who ever thought case-insensitive filesystems as a default where a good idea?) and simply does not exist on Windows (yet the build system could be extended see RInside for an existence proof, and volunteers are welcome!). See the FAQ vignette on how to add it to your PATH. A few examples are highlighted at the Github repo, as well as in the examples vignette. This release brings five new example scripts and command wrappers: A number of commands were also extended or updated (see below for more). We have a new and very slick documentation website once again utilising Material for MkDocs. Last but not least the two included vignettes now use minidown and the fabulous water css theme which reduced the file sizes of the two vignettes from, respectively, 884kb and 873kb to 47kb and 15kb. Yes, that is correct. That alone brought the package file size down from 641kb to 116kb. Incredible. The NEWS file entry is below.

Changes in littler version 0.3.12 (2020-10-04)
  • Changes in examples
    • Updates to scripts tt.r, cos.r, cow.r, c4r.r, com.r
    • New script installDeps.r to install dependencies
    • Several updates tp script check.r
    • New script installBSPM.r and installRSPM.r for binary package installation (Dirk and I aki in #81)
    • New script cranIncoming.r to check in Incoming
    • New script urlUpdate.r validates URLs as R does
  • Changes in package
    • Travis CI now uses BSPM
    • A package documentation website was added
    • Vignettes now use minidown resulting in much reduced filesizes: from over 800kb to under 50kb (Dirk in #83)

My CRANberries provides a comparison to the previous release. Full details for the littler release are provided as usual at the ChangeLog page, and now also on the new package docs website. The code is available via the GitHub repo, from tarballs and now of course also from its CRAN page and via install.packages("littler"). Binary packages are available directly in Debian as well as soon via Ubuntu binaries at CRAN thanks to the tireless Michael Rutter. Comments and suggestions are welcome at the GitHub repo. If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

30 September 2020

Antoine Beaupr : Presentation tools

I keep forgetting how to make presentations. I had a list of tools in a wiki from a previous job, but that's now private and I don't see why I shouldn't share this (even if for myself!). So here it is. What's your favorite presentation tool?

Tips
  • if you have some text to present, outline keywords so that you can present your subject without reading every word
  • ideally, don't read from your slides - they are there to help people follow, not for people to read
  • even better: make your slides pretty with only a few words, or don't make slides at all
Further advice: I'm currently using Pandoc with PDF input (with a trip through LaTeX) for most slides, because PDFs are more reliable and portable than web pages. I've also used Libreoffice, Pinpoint, and S5 (through RST) in the past. I miss Pinpoint, too bad that it died. Some of my presentations are available in my GitLab.com account: See also my list of talks and presentations which I can't seem to keep up to date.

Tools

Beamer (LaTeX)
  • LaTeX class
  • Do not use directly unless you are a LaTeX expert or masochist, see Pandoc below
  • see also powerdot
  • Home page

Darkslide
  • HTML, Javascript
  • presenter notes, table of contents, Markdown, RST, Textile, themes, code samples, auto-reload
  • Home page, demo

Impress.js

Impressive
  • simply displays PDFs or images
  • page transitions, overview screen, highlighting
  • Home page

Libreoffice Impress
  • Powerpoint clone
  • Makes my life miserable
  • PDF export, presenter notes, outline view, etc
  • Home page, screenshots

Magicpoint
  • ancestor of everyone else (1997!)
  • text input format, image support, talk timer, slide guides, HTML/Postscript export, draw on slides, X11 output
  • no release since 2008
  • Home page

mdp and lookatme (commandline)

Pandoc
  • Allows converting from basically whatever into slides, including Beamer, DZSlides, reveal.js, slideous, slidy, Powerpoint
  • PDF, HTML, Powerpoint export, presentation notes, full screen background images
  • nice plain text or markdown input format
  • Home page, documentation

PDF Presenter
  • PDF presentation tool, shows presentation notes
  • basically "Keynote for Linux"
  • Home page, pdf-presenter-console in Debian

Pinpoint
  • Native GNOME app
  • Full screen slides, PDF export, live change, presenter notes, pango markup, video, image backgrounds
  • Home page
  • Abandoned since at least 2019

Reveal.js
  • HTML, Javascript
  • PDF export, Markdown, LaTeX support, syntax-highlighting, nested slides, speaker notes
  • Source code, demo

S5
  • HTML, CSS
  • incremental, bookmarks, keyboard controls
  • can be transformed from ReStructuredText (RST) with rst2s5 with python-docutils
  • Home page, demo

sent
  • X11 only
  • plain text, black on white, image support, and that's it
  • from the suckless.org elitists
  • Home page

Sozi
  • Entire presentation is one poster, zooming and jumping around
  • SVG + Javascript
  • Home page, demo

Other options Another option I have seriously considered is just generate a series of images with good resolution, hopefully matching the resolution (or at least aspect ratio) of the output device. Then you flip through a series of images one by one. In that case, any of those image viewers (not an exhaustive list) would work: Update: it turns out I already wrote a somewhat similar thing when I did a recent presentation. If you're into rants, you might enjoy the README file accompanying the Kubecon rant presentation. TL;DR: "makes me want to scream" and "yet another unsolved problem space, sigh" (refering to "display images full-screen" specifically).

6 September 2020

Russ Allbery: September haul

So many good books, so little reading time. Jairus Banaji A Brief History of Commercial Capitalism (nonfiction)
Steven Brust The Baron of Magister Valley (sff)
Micaiah Johnson The Space Between Worlds (sff)
Ian McDonald Luna: New Moon (sff)
Elizabeth Moon Trading in Danger (sff)
Tamsyn Muir Harrow the Ninth (sff)
Suzanne Palmer Finder (sff)
Kit Rocha Beyond Shame (sff)
Kit Rocha Beyond Control (sff)
Kit Rocha Beyond Pain (sff)
Arundhati Roy Azadi (nonfiction)
Jeff VanderMeer Authority (sff)
Jeff VanderMeer Acceptance (sff)
K.B. Wagers Behind the Throne (sff)
Jarrett Walker Human Transit (nonfiction) I took advantage of a few sales to get books I know I'm going to want to read eventually for a buck or two.

26 June 2020

Dirk Eddelbuettel: littler 0.3.11: docopt updates

max-heap image The twelveth release of littler as a CRAN package is now available, following in the fourteen-ish year history as a package started by Jeff in 2006, and joined by me a few weeks later. littler is the first command-line interface for R as it predates Rscript. It allows for piping as well for shebang scripting via #!, uses command-line arguments more consistently and still starts faster. It also always loaded the methods package which Rscript only started to do in recent years. littler lives on Linux and Unix, has its difficulties on macOS due to yet-another-braindeadedness there (who ever thought case-insensitive filesystems as a default where a good idea?) and simply does not exist on Windows (yet the build system could be extended see RInside for an existence proof, and volunteers are welcome!). See the FAQ vignette on how to add it to your PATH. A few examples are highlighted at the Github repo, as well as in the examples vignette. This release mostly responds to the recent docopt release 0.7.0 which brought a breaking change for quoted arguments. In short, it is for the better because an option --as-cran is now available parsed as opt$as_cran which is easier than the earlier form where we needed to back-tick protect as-cran containing an underscore. We also added a new portmanteau-ish option to roxy.r. The NEWS file entry is below.

Changes in littler version 0.3.11 (2020-06-26)
  • Changes in examples
    • Scripts check.r and rcc.r updated to reflect updated docopt 0.7.0 behaviour of quoted arguments
    • The roxy.r script has a new ease-of-use option -f --full regrouping two other options.

CRANberries provides a comparison to the previous release. Full details for the littler release are provided as usual at the ChangeLog page. The code is available via the GitHub repo, from tarballs and now of course also from its CRAN page and via install.packages("littler"). Binary packages are available directly in Debian as well as soon via Ubuntu binaries at CRAN thanks to the tireless Michael Rutter. Comments and suggestions are welcome at the GitHub repo. If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

3 June 2020

Dirk Eddelbuettel: littler 0.3.10: Some more updates

max-heap image The eleventh release of littler as a CRAN package is now available, following in the fourteen-ish year history as a package started by Jeff in 2006, and joined by me a few weeks later. littler is the first command-line interface for R as it predates Rscript. It allows for piping as well for shebang scripting via #!, uses command-line arguments more consistently and still starts faster. It also always loaded the methods package which Rscript only started to do in recent years. littler lives on Linux and Unix, has its difficulties on macOS due to yet-another-braindeadedness there (who ever thought case-insensitive filesystems as a default where a good idea?) and simply does not exist on Windows (yet the build system could be extended see RInside for an existence proof, and volunteers are welcome!). See the FAQ vignette on how to add it to your PATH. A few examples are highlighted at the Github repo, as well as in the examples vignette. This release adds a new helper / example script installBioc.r for BioConductor package installation, generalizes the roxygenize() wrapper roxy.r a little, and polished a couple of other corners. The NEWS file entry is below.

Changes in littler version 0.3.10 (2020-06-02)
  • Changes in examples
    • The update.r script only considers writeable directories.
    • The rcc.r script tries to report full logs by setting _R_CHECK_TESTS_NLINES_=0.
    • The tt.r script has an improved ncpu fallback.
    • Several installation and updating scripts set _R_SHLIB_STRIP_ to TRUE.
    • A new script installBioc.r was added.
    • The --error option to install2.r was generalized (Sergio Oller in #78).
    • The roxy.r script was extended a little.
  • Changes in package
    • Travis CI now uses R 4.0.0 and the bionic distro

CRANberries provides a comparison to the previous release. Full details for the littler release are provided as usual at the ChangeLog page. The code is available via the GitHub repo, from tarballs and now of course also from its CRAN page and via install.packages("littler"). Binary packages are available directly in Debian as well as soon via Ubuntu binaries at CRAN thanks to the tireless Michael Rutter. Comments and suggestions are welcome at the GitHub repo. If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

16 May 2020

Russ Allbery: rra-c-util 8.2

This release of my general utility libraries and support code includes a large grab bag of fixes and improvements. portable/system.h now defines explicit_bzero in terms of memset if it is not available. The memset version is unlikely to have the same security properties since the compiler may optimize it away, but that allows me to use explicit_bzero to erase security data where it is available. For packages with Kerberos tests, generating a test krb5.conf file now works properly even if the system krb5.conf file does not set a default realm, and a krb5.conf file dropped into the test configuration directory now works properly. Thanks to Jeffrey Hutzelman for the latter fix. For packages with PAM modules, the ENTRY and EXIT logging macros can now be used like function calls, and portable/pam.h now defines PAM_MAX_RESP_SIZE if it isn't defined. Header ordering in some of the portability socket code has been restored to compatibility with a few ancient UNIX systems. This was accidentally broken by the clang-format reformatting. Thanks to Julien LIE for the fix. A few bugs in the test for SPDX license identifiers have been fixed. Finally, this release fixes warnings with Clang 10 and GCC 10. You can get the latest release from the rra-c-util distribution page.

11 May 2020

Gunnar Wolf: Certified printer fumes

After losing a fair bit of hair due to quality and reliability issues with our home laser multifunctional (Brother DCP1600-series, which we bought after checking it was meant to work on Linux And it does, but with a very buggy, proprietary driver Besides being the printer itself of quite low quality), we decided it was time to survey the market again, and get a color inkjet printer. I was not very much an enthusiast of the idea, until I found all of the major manufacturers now offer refillable ink tanks instead of the darn expensive cartridges of past decades. Lets see how it goes! Of course, with over 20 years of training, I did my homework. I was about to buy an Epson printer, but decided for an HP Ink Tank 410 Wireless printer. The day it arrived, not wanting to fuss around too much to get to see the results, I connected it to my computer using the USB cable. Everything ran smoothly and happily! No driver hunting needed, print quality is superb I hope, years from now, we stay with this impression. Next day, I tried to print over WiFi. Of course, it requires configuration. And, of course, configuration strongly wants you to do it from a Windows or MacOS machine which I don t have. OK, fall back to Android For which an app download is required (and does not thrill me, but what can I say. Oh and the app needs location services to even run. Why Maybe because it interacts with the wireless network in WiFi Direct, non-authenticated way?) Anyway, things seem to work. But they don t My computers can identify and connect with the printer from CUPS, but nothing ever comes out. Printer paused, they say. Entering the printer s web interface is somewhat ambiguous Following the old HP practices, I tried http://192.168.1.75:9100/ (no point in hiding my internal IP), and got a partial webpage sometimes (and nothing at all othertimes). Seeing the printer got detected over ipps://, my immediate reaction was to try pointing the browser to port 631. Seems to work! Got some odd messages But it seems I ll soon debug the issue away. I am not a familiar meddler in the dark lands of cups, our faithful print server, but I had to remember my toolkit.. 
# cupsenable HP_Ink_Tank_Wireless_410_series_C37468_ --release
Sucess in enabling, but self-auto-disabled right away lpstat -t was not more generous, reporting only it was still paused. Some hours later (mix in attending kids and whatnot), I finally remember to try cupsctl --debug-logging, and magically, /var/log/cups/error_log turns from being quiet to being quite chatty. And, of course, my first print job starts being processed: 
D [10/May/2020:23:07:20 -0500] Report: jobs-active=1
(...)
D [10/May/2020:23:07:25 -0500] [Job 174] Start rendering...
(...)
D [10/May/2020:23:07:25 -0500] [Job 174] STATE: -connecting-to-device
(...)
Everything looks fine and dandy so far! But, hey, given nothing came out of the printer keep reading one more second of logs (a couple dozen lines) 
D [10/May/2020:23:07:26 -0500] [Job 174] Connection is encrypted.
D [10/May/2020:23:07:26 -0500] [Job 174] Credentials are expired (Credentials have expired.)
D [10/May/2020:23:07:26 -0500] [Job 174] Printer credentials: HPC37468 / Thu, 01 Jan 1970 00:00:00 GMT / 28A59EF511A480A34798B6712DEEAE74
D [10/May/2020:23:07:26 -0500] [Job 174] No stored credentials.
D [10/May/2020:23:07:26 -0500] [Job 174] update_reasons(attr=0(), s=\"-cups-pki-invalid,cups-pki-changed,cups-pki-expired,cups-pki-unknown\")
D [10/May/2020:23:07:26 -0500] [Job 174] STATE: -cups-pki-expired
(...)
D [10/May/2020:23:08:00 -0500] [Job 174] envp[16]="CUPS_ENCRYPTION=IfRequested"
(...)
D [10/May/2020:23:08:00 -0500] [Job 174] envp[27]="PRINTER_STATE_REASONS=cups-pki-expired"
My first stabs were attempts to get CUPS not to care about expired certificates, but it seems to have been hidden or removed from its usual place. Anyway, I was already frustrated. WTF Well, yes, turns out that from the Web interface, I paid some attention to this the first time around, but let it pass (speaks wonders about my security practices!): Way, way, way too expired cert So, the self-signed certificate the printer issued at itself expired 116 years before even being issued. (is this maybe a Y2k38 bug? Sounds like it!) Interesting thing, my CUPS log mentions the printer credentials to expire at the beginning of the Unix Epoch (01 Jan 1970 00:00:00 GMT). OK, lets clickety-click away on the Web interface Didn t take me long to get to Network Advanced settings Certificates: Can manage certs! However, clicking on Configure leads me to the not very reassuring Way, way, way too expired cert I don t remember what I did for the next couple of minutes. Kept fuming Until I parsed again the output of lpstat -t, and found that: 
# lpstat -t
(...)
device for HP_Ink_Tank_Wireless_410_series_C37468_: ipps://HPF43909C37468.local:443/ipp/print
(...)
Hmmmm CUPS is connecting using good ol port 443, as if it were a Web thingy What if I do the same? Now we are talking! Click on New self-signed certificate , click on Next, a couple of reloads And a very nice color print came out of the printer, yay! Now, it still baffles me (of course I checked!): The self-signed certificate is now said to come from Issuer : CN=HPC37468, L=Vancouver, ST=Washington, C=US, O=HP,OU=HP-IPG, alright not that it matters (I can import a more meaningful one if I really feel like it), but, why is it Issued On: 2019-06-14 and set to Expires On: 2029-06-11? Anyway, print quality is quite nice. I hope to keep the printer long enough to rant at the certificate being expired in the future!

Comments Jeff Epler (Adafruit) 2020-05-11 20:39:17 -0500 why is it Issued On: 2019-06-14 and set to Expires On: 2029-06-11? Because it s 3650 days Gunnar Wolf 2020-05-11 20:39:17 -0500 Nice catch! Thanks for doing the head-scratching for me

6 March 2020

Reproducible Builds: Reproducible Builds in February 2020

Welcome to the February 2020 report from the Reproducible Builds project. One of the original promises of open source software is that distributed peer review and transparency of process results in enhanced end-user security. However, whilst anyone may inspect the source code of free and open source software for malicious flaws, almost all software today is distributed as pre-compiled binaries. This allows nefarious third-parties to compromise systems by injecting malicious code into ostensibly secure software during the various compilation and distribution processes. The motivation behind the reproducible builds effort is to provide the ability to demonstrate these binaries originated from a particular, trusted, source release: if identical results are generated from a given source in all circumstances, reproducible builds provides the means for multiple third-parties to reach a consensus on whether a build was compromised via distributed checksum validation or some other scheme. In this month s report, we cover:

If you are interested in contributing to the project, please visit our Contribute page on our website.

Media coverage & upstream news Omar Navarro Leija, a PhD student at the University Of Pennsylvania, published a paper entitled Reproducible Containers that describes in detail the workings of a new user-space container tool called DetTrace:
All computation that occurs inside a DetTrace container is a pure function of the initial filesystem state of the container. Reproducible containers can be used for a variety of purposes, including replication for fault-tolerance, reproducible software builds and reproducible data analytics. We use DetTrace to achieve, in an automatic fashion, reproducibility for 12,130 Debian package builds, containing over 800 million lines of code, as well as bioinformatics and machine learning workflows.
There was also considerable discussion on our mailing list regarding this research and a presentation based on the paper will occur at the ASPLOS 2020 conference between March 16th 20th in Lausanne, Switzerland. The many virtues of Reproducible Builds were touted as benefits for software compliance in a talk at FOSDEM 2020, debating whether the Careful Inventory of Licensing Bill of Materials Have Impact of FOSS License Compliance which pitted Jeff McAffer and Carol Smith against Bradley Kuhn and Max Sills. (~47 minutes in). Nobuyoshi Nakada updated the canonical implementation of the Ruby programming language a change such that filesystem globs (ie. calls to list the contents of filesystem directories) will henceforth be sorted in ascending order. Without this change, the underlying nondeterministic ordering of the filesystem is exposed to the language which often results in an unreproducible build. Vagrant Cascadian reported on our mailing list regarding a quick reproducible test for the GNU Guix distribution, which resulted in 81.9% of packages registering as reproducible in his installation: 
$ guix challenge --verbose --diff=diffoscope ...
2,463 store items were analyzed:
  - 2,016 (81.9%) were identical
  - 37 (1.5%) differed
  - 410 (16.6%) were inconclusive
Jeremiah Orians announced on our mailing list the release of a number of tools related to cross-compilation such as M2-Planet and mescc-tools-seed. This project attemps a full bootstrap of a cross-platform compiler for the C programming language (written in C itself) from hex, the ultimate goal being able to demonstrate fully-bootstrapped compiler from hex to the GCC GNU Compiler Collection. This has many implications in and around Ken Thompson s Trusting Trust attack outlined in Thompson s 1983 Turing Award Lecture. Twitter user @TheYoctoJester posted an executive summary of reproducible builds in the Yocto Project: Finally, Reddit user tofflos posted to the /r/Java subreddit asking about how to achieve reproducible builds with Maven and Chris Lamb noticed that the Linux kernel documentation about reproducible builds of it is available on the kernel.org homepages in an attractive HTML format.

Distribution work

Debian Chris Lamb created a merge request for the core debian-installer package to allow all arguments and options from sources.list files (such as [check-valid-until=no] , etc.) in order that we can test the reproducibility of the installer images on the Reproducible Builds own testing infrastructure. (#13) Thorsten Glaser followed-up to a bug filed against the dpkg-source component that was originally filed in late 2015 that claims that the build tool does not respect permissions when unpacking tarballs if the umask is set to 0002. Matthew Garrett posted to the debian-devel mailing list on the topic of Producing verifiable initramfs images as part of a wider conversation on being able to trust the entire software stack on our computers. 59 reviews of Debian packages were added, 30 were updated and 42 were removed this month adding to our knowledge about identified issues. Many issue types were noticed and categorised by Chris Lamb, including:

openSUSE In openSUSE, Bernhard M. Wiedemann published his monthly Reproducible Builds status update as well as provided the following patches:

Software development

diffoscope diffoscope is our in-depth and content-aware diff-like utility that can locate and diagnose reproducibility issues. It is run countless times a day on our testing infrastructure and is essential for identifying fixes and causes of nondeterministic behaviour. Chris Lamb made the following changes this month, including uploading version 137 to Debian:
  • The sng image utility appears to return with an exit code of 1 if there are even minor errors in the file. (#950806)
  • Also extract classes2.dex, classes3.dex from .apk files extracted by apktool. (#88)
  • No need to use str.format if we are just returning the string. [ ]
  • Add generalised support for ignoring returncodes [ ] and move special-casing of returncodes in zip to use Command.VALID_RETURNCODES. [ ]

Other tools disorderfs is our FUSE-based filesystem that deliberately introduces non-determinism into directory system calls in order to flush out reproducibility issues. This month, Vagrant Cascadian updated the Vcs-Git to specify the debian packaging branch. [ ] reprotest is our end-user tool to build same source code twice in widely differing environments and then checks the binaries produced by each build for any differences. This month, versions 0.7.13 and 0.7.14 were uploaded to Debian unstable by Holger Levsen after Vagrant Cascadian added support for GNU Guix [ ].

Project documentation & website There was more work performed on our documentation and website this month. Bernhard M. Wiedemann added a Java Gradle Build Tool snippet to the SOURCE_DATE_EPOCH documentation [ ] and normalised various terms to unreproducible [ ]. Chris Lamb added a Meson.build example [ ] and improved the documentation for the CMake [ ] to the SOURCE_DATE_EPOCH documentation, replaced anyone can with anyone may as, well, not everyone has the resources, skills, time or funding to actually do what it refers to [ ] and improved the pre-processing for our report generation [ ][ ][ ][ ] etc. In addition, Holger Levsen updated our news page to improve the list of reports [ ], added an explicit mention of the weekly news time span [ ] and reverted sorting of news entries to have latest on top [ ] and Mattia Rizzolo added Codethink as a non-fiscal sponsor [ ] and lastly Tianon Gravi added a Docker Images link underneath the Debian project on our Projects page [ ].

Upstream patches The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including: Vagrant Cascadian submitted patches via the Debian bug tracking system targeting the packages the Civil Infrastructure Platform has identified via the CIP and CIP build depends package sets:

Testing framework We operate a fully-featured and comprehensive Jenkins-based testing framework that powers tests.reproducible-builds.org. This month, the following changes were made by Holger Levsen: In addition, Mattia Rizzolo added an Apache web server redirect for buildinfos.debian.net [ ] and reverted the reshuffling of arm64 architecture builders [ ]. The usual build node maintenance was performed by Holger Levsen, Mattia Rizzolo [ ][ ] and Vagrant Cascadian.

Getting in touch If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

This month s report was written by Bernhard M. Wiedemann, Chris Lamb and Holger Levsen. It was subsequently reviewed by a bunch of Reproducible Builds folks on IRC and the mailing list.

8 November 2017

Dirk Eddelbuettel: R / Finance 2018 Call for Papers

The tenth (!!) annual annual R/Finance conference will take in Chicago on the UIC campus on June 1 and 2, 2018. Please see the call for papers below (or at the website) and consider submitting a paper. We are once again very excited about our conference, thrilled about who we hope may agree to be our anniversary keynotes, and hope that many R / Finance users will not only join us in Chicago in June -- and also submit an exciting proposal. So read on below, and see you in Chicago in June!

Call for Papers R/Finance 2018: Applied Finance with R
June 1 and 2, 2018
University of Illinois at Chicago, IL, USA The tenth annual R/Finance conference for applied finance using R will be held June 1 and 2, 2018 in Chicago, IL, USA at the University of Illinois at Chicago. The conference will cover topics including portfolio management, time series analysis, advanced risk tools, high-performance computing, market microstructure, and econometrics. All will be discussed within the context of using R as a primary tool for financial risk management, portfolio construction, and trading. Over the past nine years, R/Finance has includedattendeesfrom around the world. It has featured presentations from prominent academics and practitioners, and we anticipate another exciting line-up for 2018. We invite you to submit complete papers in pdf format for consideration. We will also consider one-page abstracts (in txt or pdf format) although more complete papers are preferred. We welcome submissions for both full talks and abbreviated "lightning talks." Both academic and practitioner proposals related to R are encouraged. All slides will be made publicly available at conference time. Presenters are strongly encouraged to provide working R code to accompany the slides. Data sets should also be made public for the purposes of reproducibility (though we realize this may be limited due to contracts with data vendors). Preference may be given to presenters who have released R packages. Please submit proposals online at http://go.uic.edu/rfinsubmit. Submissions will be reviewed and accepted on a rolling basis with a final submission deadline of February 2, 2018. Submitters will be notified via email by March 2, 2018 of acceptance, presentation length, and financial assistance (if requested). Financial assistance for travel and accommodation may be available to presenters. Requests for financial assistance do not affect acceptance decisions. Requests should be made at the time of submission. Requests made after submission are much less likely to be fulfilled. Assistance will be granted at the discretion of the conference committee. Additional details will be announced via the conference website at http://www.RinFinance.com/ as they become available. Information on previous years'presenters and their presentations are also at the conference website. We will make a separate announcement when registration opens. For the program committee:
Gib Bassett, Peter Carl, Dirk Eddelbuettel, Brian Peterson,
Dale Rosenthal, Jeffrey Ryan, Joshua Ulrich

13 August 2017

Dirk Eddelbuettel: RProtoBuf 0.4.10

RProtoBuf provides R bindings for the Google Protocol Buffers ("ProtoBuf") data encoding and serialization library used and released by Google, and deployed fairly widely in numerous projects as a language and operating-system agnostic protocol. A new releases RProtoBuf 0.4.10 just appeared on CRAN. It is a maintenance releases replacing one leftover errorenous use of package= in .Call with the correct PACKAGE= (as requsted by CRAN). It also integrates a small robustification in the deserializer when encountering invalide objects; this was both reported and fixed by Jeffrey Shen.

Changes in RProtoBuf version 0.4.10 (2017-08-13)
  • More careful operation in deserializer checking for a valid class attribute (Jeffrey Shen in #29 fixing #28)
  • At the request of CRAN, correct one .Call() argument to PACKAGE=; update routine registration accordingly

CRANberries also provides a diff to the previous release. The RProtoBuf page has an older package vignette, a 'quick' overview vignette, a unit test summary vignette, and the pre-print for the JSS paper. Questions, comments etc should go to the GitHub issue tracker off the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

29 July 2017

Dirk Eddelbuettel: Updated overbought/oversold plot function

A good six years ago I blogged about plotOBOS() which charts a moving average (from one of several available variants) along with shaded standard deviation bands. That post has a bit more background on the why/how and motivation, but as a teaser here is the resulting chart of the SP500 index (with ticker ^GSCP): Example chart of overbought/oversold levels from plotOBOS() function The code uses a few standard finance packages for R (with most of them maintained by Joshua Ulrich given that Jeff Ryan, who co-wrote chunks of these, is effectively retired from public life). Among these, xts had a recent release reflecting changes which occurred during the four (!!) years since the previous release, and covering at least two GSoC projects. With that came subtle API changes: something we all generally try to avoid but which is at times the only way forward. In this case, the shading code I used (via polygon() from base R) no longer cooperated with the beefed-up functionality of plot.xts(). Luckily, Ross Bennett incorporated that same functionality into a new function addPolygon --- which even credits this same post of mine. With that, the updated code becomes 
## plotOBOS -- displaying overbough/oversold as eg in Bespoke's plots
##
## Copyright (C) 2010 - 2017  Dirk Eddelbuettel
##
## This is free software: you can redistribute it and/or modify it
## under the terms of the GNU General Public License as published by
## the Free Software Foundation, either version 2 of the License, or
## (at your option) any later version.
suppressMessages(library(quantmod))     # for getSymbols(), brings in xts too
suppressMessages(library(TTR))          # for various moving averages
plotOBOS <- function(symbol, n=50, type=c("sma", "ema", "zlema"),
                     years=1, blue=TRUE, current=TRUE, title=symbol,
                     ticks=TRUE, axes=TRUE)  
    today <- Sys.Date()
    if (class(symbol) == "character")  
        X <- getSymbols(symbol, from=format(today-365*years-2*n), auto.assign=FALSE)
        x <- X[,6]                          # use Adjusted
      else if (inherits(symbol, "zoo"))  
        x <- X <- as.xts(symbol)
        current <- FALSE                # don't expand the supplied data
     
    n <- min(nrow(x)/3, 50)             # as we may not have 50 days
    sub <- ""
    if (current)  
        xx <- getQuote(symbol)
        xt <- xts(xx$Last, order.by=as.Date(xx$ Trade Time ))
        colnames(xt) <- paste(symbol, "Adjusted", sep=".")
        x <- rbind(x, xt)
        sub <- paste("Last price: ", xx$Last, " at ",
                     format(as.POSIXct(xx$ Trade Time ), "%H:%M"), sep="")
     
    type <- match.arg(type)
    xd <- switch(type,                  # compute xd as the central location via selected MA smoother
                 sma = SMA(x,n),
                 ema = EMA(x,n),
                 zlema = ZLEMA(x,n))
    xv <- runSD(x, n)                   # compute xv as the rolling volatility
    strt <- paste(format(today-365*years), "::", sep="")
    x  <- x[strt]                       # subset plotting range using xts' nice functionality
    xd <- xd[strt]
    xv <- xv[strt]
    xyd <- xy.coords(.index(xd),xd[,1]) # xy coordinates for direct plot commands
    xyv <- xy.coords(.index(xv),xv[,1])
    n <- length(xyd$x)
    xx <- xyd$x[c(1,1:n,n:1)]           # for polygon(): from first point to last and back
    if (blue)  
        blues5 <- c("#EFF3FF", "#BDD7E7", "#6BAED6", "#3182BD", "#08519C") # cf brewer.pal(5, "Blues")
        fairlylight <<- rgb(189/255, 215/255, 231/255, alpha=0.625) # aka blues5[2]
        verylight <<- rgb(239/255, 243/255, 255/255, alpha=0.625) # aka blues5[1]
        dark <<- rgb(8/255, 81/255, 156/255, alpha=0.625) # aka blues5[5]
        ## buglet in xts 0.10-0 requires the <<- here
      else  
        fairlylight <<- rgb(204/255, 204/255, 204/255, alpha=0.5)  # two suitable grays, alpha-blending at 50%
        verylight <<- rgb(242/255, 242/255, 242/255, alpha=0.5)
        dark <<- 'black'
     
    plot(x, ylim=range(range(x, xd+2*xv, xd-2*xv, na.rm=TRUE)), main=title, sub=sub, 
         major.ticks=ticks, minor.ticks=ticks, axes=axes) # basic xts plot setup
    addPolygon(xts(cbind(xyd$y+xyv$y, xyd$y+2*xyv$y), order.by=index(x)), on=1, col=fairlylight)  # upper
    addPolygon(xts(cbind(xyd$y-xyv$y, xyd$y+1*xyv$y), order.by=index(x)), on=1, col=verylight)    # center
    addPolygon(xts(cbind(xyd$y-xyv$y, xyd$y-2*xyv$y), order.by=index(x)), on=1, col=fairlylight)  # lower
    lines(xd, lwd=2, col=fairlylight)   # central smooted location
    lines(x, lwd=3, col=dark)           # actual price, thicker
and the main change are the three calls to addPolygon. To illustrate, we call plotOBOS("SPY", years=2) with an updated plot of the ETF representing the SP500 over the last two years: Updated example chart of overbought/oversold levels from plotOBOS() function Comments and further enhancements welcome!

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

15 July 2017

Dirk Eddelbuettel: Rcpp 0.12.12: Rounding some corners

The twelveth update in the 0.12.* series of Rcpp landed on CRAN this morning, following two days of testing at CRAN preceded by five full reverse-depends checks we did (and which are always logged in this GitHub repo). The Debian package has been built and uploaded; Windows and macOS binaries should follow at CRAN as usual. This 0.12.12 release follows the 0.12.0 release from late July, the 0.12.1 release in September, the 0.12.2 release in November, the 0.12.3 release in January, the 0.12.4 release in March, the 0.12.5 release in May, the 0.12.6 release in July, the 0.12.7 release in September, the 0.12.8 release in November, the 0.12.9 release in January, the 0.12.10.release in March, and the 0.12.11.release in May making it the sixteenth release at the steady and predictable bi-montly release frequency. Rcpp has become the most popular way of enhancing GNU R with C or C++ code. As of today, 1097 packages (and hence 71 more since the last release in May) on CRAN depend on Rcpp for making analytical code go faster and further, along with another 91 in BioConductor. This releases contain a fairly large number of fairly small and focused pull requests most of which either correct some corner cases or improve other aspects. JJ tirelessly improved the package registration added in the previous release and following R 3.4.0. Kirill tidied up a number of small issues allowing us to run compilation in even more verbose modes---usually a good thing. Jeroen, Elias Pipping and Yo Gong all contributed as well, and we thank everybody for their contributions. All changes are listed below in some detail.

Changes in Rcpp version 0.12.12 (2017-07-13)
  • Changes in Rcpp API:
    • The tinyformat.h header now ends in a newline (#701).
    • Fixed rare protection error that occurred when fetching stack traces during the construction of an Rcpp exception (Kirill M ller in #706).
    • Compilation is now also possibly on Haiku-OS (Yo Gong in #708 addressing #707).
    • Dimension attributes are explicitly cast to int (Kirill M ller in #715).
    • Unused arguments are no longer declared (Kirill M ller in #716).
    • Visibility of exported functions is now supported via the R macro atttribute_visible (Jeroen Ooms in #720).
    • The no_init() constructor accepts R_xlen_t (Kirill M ller in #730).
    • Loop unrolling used R_xlen_t (Kirill M ller in #731).
    • Two unused-variables warnings are now avoided (Jeff Pollock in #732).
  • Changes in Rcpp Attributes:
    • Execute tools::package_native_routine_registration_skeleton within package rather than current working directory (JJ in #697).
    • The R portion no longer uses dir.exists to no require R 3.2.0 or newer (Elias Pipping in #698).
    • Fix native registration for exports with name attribute (JJ in #703 addressing #702).
    • Automatically register init functions for Rcpp Modules (JJ in #705 addressing #704).
    • Add Shield around parameters in Rcpp::interfaces (JJ in #713 addressing #712).
    • Replace dot (".") with underscore ("_") in package names when generating native routine registrations (JJ in #722 addressing #721).
    • Generate C++ native routines with underscore ("_") prefix to avoid exporting when standard exportPattern is used in NAMESPACE (JJ in #725 addressing #723).

Thanks to CRANberries, you can also look at a diff to the previous release. As always, even fuller details are on the Rcpp Changelog page and the Rcpp page which also leads to the downloads page, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

8 July 2017

Daniel Silverstone: Gitano - Approaching Release - Access Control Changes

As mentioned previously I am working toward getting Gitano into Stretch. A colleague and friend of mine (Richard Maw) did a large pile of work on Lace to support what we are calling sub-defines. These let us simplify Gitano's ACL files, particularly for individual projects. In this posting, I'd like to cover what has changed with the access control support in Gitano, so if you've never used it then some of this may make little sense. Later on, I'll be looking at some better user documentation in conjunction with another friend of mine (Lars Wirzenius) who has promised to help produce a basic administration manual before Stretch is totally frozen.

Sub-defines With a more modern lace (version 1.3 or later) there is a mechanism we are calling 'sub-defines'. Previously if you wanted to write a ruleset which said something like "Allow Steve to read my repository" you needed: 
define is_steve user exact steve
allow "Steve can read my repo" is_steve op_read
And, as you'd expect, if you also wanted to grant read access to Jeff then you'd need yet set of defines: 
define is_jeff user exact jeff
define is_steve user exact steve
define readers anyof is_jeff is_steve
allow "Steve and Jeff can read my repo" readers op_read
This, while flexible (and still entirely acceptable) is wordy for small rulesets and so we added sub-defines to create this syntax: 
allow "Steve and Jeff can read my repo" op_read [anyof [user exact jeff] [user exact steve]]
Of course, this is generally neater for simpler rules, if you wanted to add another user then it might make sense to go for: 
define readers anyof [user exact jeff] [user exact steve] [user exact susan]
allow "My friends can read my repo" op_read readers
The nice thing about this sub-define syntax is that it's basically usable anywhere you'd use the name of a previously defined thing, they're compiled in much the same way, and Richard worked hard to get good error messages out from them just in case.

No more auto_user_XXX and auto_group_YYY As a result of the above being implemented, the support Gitano previously grew for automatically defining users and groups has been removed. The approach we took was pretty inflexible and risked compilation errors if a user was deleted or renamed, and so the sub-define approach is much much better. If you currently use auto_user_XXX or auto_group_YYY in your rulesets then your upgrade path isn't bumpless but it should be fairly simple:
  1. Upgrade your version of lace to 1.3
  2. Replace any auto_user_FOO with [user exact FOO] and similarly for any auto_group_BAR to [group exact BAR].
  3. You can now upgrade Gitano safely.

No more 'basic' matches Since Gitano first gained support for ACLs using Lace, we had a mechanism called 'simple match' for basic inputs such as groups, usernames, repo names, ref names, etc. Simple matches looked like user FOO or group !BAR. The match syntax grew more and more arcane as we added Lua pattern support refs ~^refs/heads/$ user /. When we wanted to add proper PCRE regex support we added a syntax of the form: user pcre ^/.+?... where pcre could be any of: exact, prefix, suffix, pattern, or pcre. We had a complex set of rules for exactly what the sigils at the start of the match string might mean in what order, and it was getting unwieldy. To simplify matters, none of the "backward compatibility" remains in Gitano. You instead MUST use the what how with match form. To make this slightly more natural to use, we have added a bunch of aliases: is for exact, starts and startswith for prefix, and ends and endswith for suffix. In addition, kind of match can be prefixed with a ! to invert it, and for natural looking rules not is an alias for !is. This means that your rulesets MUST be updated to support the more explicit syntax before you update Gitano, or else nothing will compile. Fortunately this form has been supported for a long time, so you can do this in three steps.
  1. Update your gitano-admin.git global ruleset. For example, the old form of the defines used to contain define is_gitano_ref ref ~^refs/gitano/ which can trivially be replaced with: define is_gitano_ref ref prefix refs/gitano/
  2. Update any non-zero rulesets your projects might have.
  3. You can now safely update Gitano
If you want a reference for making those changes, you can look at the Gitano skeleton ruleset which can be found at https://git.gitano.org.uk/gitano.git/tree/skel/gitano-admin/rules/ or in /usr/share/gitano if Gitano is installed on your local system. Next time, I'll likely talk about the deprecated commands which are no longer in Gitano, and how you'll need to adjust your automation to use the new commands.

14 May 2017

Bits from Debian: New Debian Developers and Maintainers (March and April 2017)

The following contributors got their Debian Developer accounts in the last two months: The following contributors were added as Debian Maintainers in the last two months: Congratulations!

Next.

Previous.